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Background The large treatment gap for people suffering from mental disorders 
has led to initiatives to scale up mental health services. In order to 
track progress, estimates of programme coverage, and changes in 
coverage over time, are needed. 

Methods Systematic review of mental health programme evaluations that 
assess coverage, measured either as the proportion of the target popu- 
lation in contact with services (contact coverage) or as the proportion 
of the target population who receive appropriate and effective care 
(effective coverage). We performed a search of electronic databases 
and grey literature up to March 2013 and contacted experts in the 
field. Methods to estimate the numerator (service utilization) and 
the denominator (target population) were reviewed to explore meth- 
ods which could be used in programme evaluations. 

Results We identified 15 735 unique records of which only seven met the 
inclusion criteria. All studies reported contact coverage. No study 
explicitly measured effective coverage, but it was possible to esti- 
mate this for one study. In six studies the numerator of coverage, 
service utilization, was estimated using routine clinical information, 
whereas one study used a national community survey. The methods 
for estimating the denominator, the population in need of services, 
were more varied and included national prevalence surveys case 
registers, and estimates from the literature. 

Conclusions Very few coverage estimates are available. Coverage could be esti- 
mated at low cost by combining routine programme data with 
population prevalence estimates from national surveys. 
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Introduction 

The large disparity between the number of people esti- 
mated to suffer from mental disorders and the propor- 
tion of those who receive adequate and appropriate 
treatment for these disorders, known as the 'treatment 



gap', is a core concern of the emerging discipline of 
global mental health.^ The median global treatment 
gap is estimated to be 32.2% for schizophrenia and 
56.3% for depression.^ This has resulted in a dominant 
focus of current global mental health initiatives to scale 
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up mental health services. In order to track the pro- 
gress of regional, national and global efforts to reduce 
the treatment gap, evaluations of service need, utiliza- 
tion rates and programme coverage levels in popula- 
tions are essential. These are all measures of the 
inverse of the treatment gap: treatment coverage. 

Coverage can be conceived of as a one-dimensional 
concept such as a target for successful service scale-up 
of a particular intervention ( such as vaccination rates in 
the under -five population), or as a multi- dimensional 
concept for moving tow^ards universal access to health 
care. Multifaceted definitions of coverage include insur- 
ance against the risk of catastrophic health spending 
combined with access to an essential set of services.^ 

Intervention- specific or service coverage reflects the 
extent to which those in need of a health intervention 
get it. Individuals in need of a specific health service 
pass through a number of stages before they can re- 
ceive the full therapeutic benefit of an intervention.^ 
Specifically, the intervention needs to be: (i) physic- 
ally available; (ii) financially and geographically ac- 
cessible; (iii) acceptable; (iv) used; and (v) delivered 
appropriately and effectively. Even if physical avail- 
ability is 100%, each of the subsequent filters could 
easily reduce the preceding ratio by a third, resulting 
in a final or effective coverage of only 20%. Such 
approaches have been used to assess the coverage of 
interventions in other areas of health care, e.g. the 
coverage of mosquito nets^ and malaria treatment.'^ 

The framework for the definition of coverage used in 
this review is the five levels of service coverage de- 
veloped by Tanahashi et al7 (Figure 1, modified to 
include examples of how each level of coverage can 
be measured). The first three levels address the po- 
tential coverage of a programme, comprising the 
availability of a service, how accessible it is and how 
acceptable the treatment provided is. For service avail- 
ability (level 1), a range of tools have been developed 
and applied in a number of different national con- 
texts for the purpose of comparing the structure, 
range and supply of specialist adult mental health 
services in a catchment area. Good examples of 
these are the WHO ATLAS project^^ and the WHO 
AIMS instrument which collects detailed information 
about the mental health system of a country. 

Service accessibility (level 2) is also measured at the 
health system rather than the individual level and 
requires a separate set of evaluation methods includ- 
ing national level monitoring and evaluation systems 
(see e.g. Rotondi 2012^^) and targeted surveys (e.g. 
using the WHO AIMS instrument^ ^) to assess how 
accessible mental health services are to the population 
in domains such as geographical distribution of ser- 
vices and financial affordability. Acceptability of the 
services to potential and actual services users (level 3) 
is most often measured through mixed methods 
including qualitative interviews with service users. 
These three types of coverage have been employed 
only rarely in the field of mental health, and where 



so, do not in themselves capture the concept of con- 
tact or effective coverage as they are pre-conditions of 
coverage rather than coverage itself. 

Levels 4 and 5 of the Tanahashi framework^ meas- 
ure actual rather than potential coverage, and are the 
focus of this review. 

Contact coverage corresponds to level 4 of the 
Tanahashi framework and captures the proportion of 
persons in need of a service (e.g. the number of cases 
with a diagnosable disorder such as schizophrenia or 
depression) who receive an intervention that is appro- 
priate to their condition. It is calculated as the ratio be- 
tween the numerator of those receiving services and the 
denominator of those estimated to need those services 
in the population. Contact coverage also includes equit- 
able coverage which assesses whether the service covers 
different socio-demographic or other groups equitably. 

Effective coverage corresponds to level 5 and has 
been defined as 'the probability that individuals will 
receive health gain from an intervention if they need 
it'.^^ Effective coverage links the three concepts of 
need, utilization of services and quality of care 
received, including issues of therapeutic response as 
well as provider and patient adherence. Effective 
coverage therefore places intervention- specific cover- 
age within a health system constraints perspective. 

Effective coverage is calculated by assessing the pro- 
portion of those in need of a service who gain the 
intended health benefit from that service. This con- 
struct combines contact coverage with intervention 
effectiveness, the latter measured by tracking changes 
in clinical outcomes over the course of treatment. The 
concept of quality of care, for which a number of 
frameworks have been developed, is central to 
the concept of effective coverage. For example, the 
OECD Health Care Quality Indicators Project defines 
three domains within the concept of quality of care: 
effectiveness; safety; and responsiveness/patient 
centredness.^^ These three domains are essential com- 
ponents in establishing whether an individual receives 
the intended health gain from an intervention. 

In order for the efforts to scale up services for 
mental disorders to be tracked and evaluated, the 
extent, equity and effectiveness of the coverage of 
mental health programmes must be established. 
Here we describe a systematic review of evaluations 
of the contact coverage and effective coverage of 
mental health programmes globally. The results will 
be used to develop a framework of methods to guide 
the measurement of coverage linked to efforts to 
evaluate the scale up of services for mental, neuro- 
logical and substance use (MNS) disorders. 



Methods 

Search strategy 

The methods and results in this paper are presented 
according to the PRISMA statement for reporting 
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METHODS 



p. EFFECTIVE COVERAGE 

People who receive effective core 



Proportion of tho?;p who need treatment taken from prev^ilenre 
surveys who received a health benefit from the programme 
measured by an analysis of the impact of the programme on 
patient outcomes 



4, CONTACT COVERAGE 

People who use service 



Service utilization taken from programme records divided 
by the total population in need of services taken from 
prevalence surveys of the disorder 



3, ACCEPTABILITY COVERAGE 

People who are willing to use service 



Acreptnbillty of ?;ervice!^ to users measures using 
mixed methods including qualitative nterviews 
with potential and past service users 



2, ACCESSIBILITY COVERAGE 

People who can use service 



Equitab e distribution of services across the 
population rreasured through national level 
M&E systems and targeted surveys 



1, AVAILABILITY COVERAGE 

People for whom sen/ice is ovai labia 



Rate of services avai able in the 
population estimated from national 
level M&E systems & targeted surveys 



TOTAL TARGET POPULATION 



Figure 1 Levels of programme coverage. Figure adapted from Tahanashi 19787 M&E, monitoring and evaluation 



systematic reviews/^ A protocol for the review was 
developed and the search strategy finalized in collab- 
oration with an information scientist. We searched 
the electronic databases Medline, EMBASE, 
Psyclnfo, Global Health and Econlit in November 
2012 using Medical Subject electronic databases 
using broad search terms designed to capture all 
evaluations of mental health programmes, as many 
programmes did not include coverage terms in their 
title and abstract. The search terms therefore com- 
bined the domains mental disorders + service provi- 
sion + evaluation (for a list of full search terms see 
Supplementary Appendix A, available at IJE online). 
We manually reviewed the full text of all mental 
health programme evaluations to select only those 
which evaluated contact or effectiveness coverage. 
No restrictions were put on date of publication, but 
the search was restricted to English language publi- 
cations due to the size of the literature initially re- 
viewed. Reviews and commentaries were excluded to 
restrict the search to original reports of programme 
evaluations. 

Grey literature was identified through searching the 
NffS Evidence and Eldis databases and the websites 
of relevant non- governmental organizations (NGOs) 
and multilateral organizations involved in developing 
and delivering mental health care programmes. In all, 
54 individuals in key organizations involved in mental 
health care delivery and other experts in the field 
were contacted for additional studies. A Google 
search using search terms based on the formal 



search strategy was performed and the search results 
screened for inclusion until saturation was reached. 
Finally, a snowball search of potentially relevant ref- 
erences identified through full -text reviews was con- 
ducted. Studies identified through this strategy were 
accepted until March 2013. 

Inclusion criteria 

Table 1 lists the inclusion and exclusion criteria for 
the review. All quantitative evaluations of the contact 
and/or effective coverage of routine mental health 
care programmes delivered at scale to the general 
population were included. Evaluations of population- 
level access to mental health services were excluded if 
mental health service utilization was not linked to a 
specific mental health care programme. Any routine 
programme designed to treat any mental disorder was 
included, including single interventions (e.g. psycho- 
logical therapy), packages of care (e.g. stepped collab- 
orative care for depression) and whole mental health 
systems (e.g. community mental health teams refer- 
ring to inpatient units). Time-limited treatment pro- 
grammes implemented solely as a pilot study or as 
part of a research project were excluded. Mental 
health promotion programmes and services delivered 
to specific populations including veterans, homeless 
people and prison populations were excluded to 
focus the review on methods applicable to estimating 
the treatment coverage of the general population. 'At 
scale' was defined as being delivered in at least one 
administrative health unit determined according to 
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Table 1 Inclusion and exclusion criteria 





Included 


Excluded 


Publication type 


Any date 

/TJLiy LULlilLiy 






English language 


Non-English language 




Peer-reviewed articles 


Reviews 




Grey literature such as technical reports 


Systematic reviews/ meta-analyses 
Commentaries 


Mental health 


Any treatment programme for any MNS dis- 


Mental health care interventions that were only 


programme 


order delivered at scale as part of routine 


implemented to evaluate their effectiveness as 




health care. A programme can encompass a 


part of a research project or pilot study 




single intervention, a package of care or a 
health system 


Mental health promotion and mental illness 
prevention programmes 

Studies evaluating training programmes for 
mental health care staff delivering treatment 
interventions 

Web -based mental health treatment 
programmes 


Study population 


General population including older people, 


Specific populations including veterans, pris- 




adults, adolescents and children 


oners, armed forces and homeless people 


Setting 


All routine health care settings 


Specialist settings such as prisons, homes for 
veterans 


Scale 


Programme delivered at scale defined as at least 


Programme delivered to an area smaller than 1 




1 administrative health unit 


administrative health unit 


Study design 


Any study design reporting the quantitative re- 


Programme evaluations which do not evaluate 




sults of an evaluation of the coverage of a 


coverage 




mental health care programme 


Qualitative studies 


Methods 


Study reports the methods used to evaluate 


No methods reported or methods reported in 




coverage 


insufficient detail 


Outcome 


Measures reported of crude, effective, equitable 


Programmes which do not report a relevant 




or population coverage at the individual ser- 


measure of coverage 




vice user level. These may be reported as 






ratios, percentages or crude figures 





the country in which the programme was delivered 
(e.g. in the UK an administrative health unit is 
defined as a primary care trust). 

Selection of studies 

One author of the present study (L.L.) screened the 
titles of all database search results to remove studies 
not related to mental health care. Four authors (L.L., 
D.F., M.D.S. and S.R.) then paired up and independ- 
ently double- screened the titles of the remaining ab- 
stracts and, if necessary, the full texts to determine 
whether they met the pre -specified inclusion criteria 
outlined in Table 1. Disagreements were resolved 
through discussion with a third reviewer. 

Data synthesis 

Data were extracted by two authors (M.D.S. and L.L.) 
using a standard data extraction form including target 
population, level of evaluation, mental health 



condition, programme description, study design, the 
methods used to measure coverage and coverage re- 
sults. The methods used by included studies to esti- 
mate the numerator of service utilization and the 
denominator of target population were reviewed to 
provide a description of methods which could be 
used in programme evaluations. 



Results 

Figure 2 presents the search and selection process for 
the review. A total of 15 735 unique records were 
identified, including 21 from contacting experts in 
the field and relevant organizations, and 76 from 
the internet search and references screening. After ir- 
relevant titles were removed, 3060 records were 
double -screened against the inclusion criteria. Of 
these, 136 were evaluations of the effectiveness, 
cost-effectiveness or coverage of mental health 



ESTIMATING THE COVERAGE OF MENTAL HEALTH PROGRAMMES 345 



15 539 unique records 
identified from 
database searching 



21 records identified 
from contacting 
authors 



76 records identified 
from non-database 
searching 



15 735 unique records 
identified 



15 735 records single- 
screened for 

irrelevance based on 
abstract/title 



12 675 records excluded 



3060 titles/abstracts 
double screened 



278 full text reports 
double-screened 



2 78 3 records excluded 



142 reports excluded: 

• 79 were not programme evaluations 

• 14 were not programmes of routine care 

• 1 was a prevention programme only 

• 1 was not a programme to treat MNS 

• 4 were not programmes of treatment for the 
general population 

• 3 were not delivered at scale 

• 35 did not include methods to evaluate effect on 
patient outcomes, effectiveness or cost -effectiveness 

• 5 d id n ot i ncl ude q ua nt ita tive o utco mes 



136 reports evaluating 

mental health 
programmes identified 



^ 7 reports evaluating 
coverage of mental 
health programmes 
included 



Figure 2 Selection of studies 



129 reports excluded: 

• 109 did not report any coverage measures 

• 1 3 on ly repor ted se r vice u t i t i za t ion mea s u res 

• 7 only reported service availability measures 



programmes. Only 7 of these 136 studies evaluated 
programme coverage and were included in the 
review. The other 129 evaluated either the effect of 
the programme on patient outcomes, or the cost- 
effectiveness of the programme, and will be the 
subject of a forthcoming review. 

Five of the studies were from high-income countries 
(four from Europe and one from Australia) and two 
from upper middle-income countries (Chile and 
China). Three of the evaluations were of national 
programmes, two regional and two of single 



administrative health units (districts). The pro- 
grammes covered a range of mental disorders includ- 
ing common mental disorders, ^'^'^^ severe mental 
illness,^ ^ substance misuse^^ and a combination of 
mental disorders. ^^"^^ The treatment programmes 
were equally varied, ranging from: a national pro- 
gramme increasing access to psychological therapies 
in England; a regional methadone treatment pro- 
gramme in China;^^ the nationwide integration of de- 
pression treatment into primary care in Chile;^^ a 
national financing system to increase access to 
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mental health specialists in Australia; district-level 
community mental health services in Italy;^^'^^ and 
regional hospital-based services for people with 
severe mental illness such as schizophrenia or bipolar 
disorder in Denmark.^ ^ Only two of the studies, both 
government -commissioned reports, evaluated cover- 
age as part of a wider programme evaluation includ- 
ing the effect on patient outcomes and programme 
costs/'^'^^ All seven studies included a measure of 
contact coverage. No study explicitly measured effect- 
ive coverage, but it was possible to calculate an esti- 
mate of this for one study/^ Table 2 presents the 
characteristics of included studies. 

Methods to estimate contact coverage 

Estimation of the numerator: service utilization 

The studies used three different methods to measure 
the numerator of service utilization. Five studies 
determined service utilization from routine patient 
data collected by the programme. ^'^'^^'^^'^^ One of 
these^^ collated data collected by the programme to 
a national level in the Danish Psychiatric Central 
Register, a computerized register which includes in- 
formation about all admissions to psychiatric wards 
and outpatient contacts. The second method used a 
national routine database to measure utilization, spe- 
cifically the Medicare Benefits Schedule which records 
service utilization through payments made to medical 
practitioners for services delivered. The third 
method, used by the Chile study, used service utiliza- 
tion data not collected by the programme.^^ This na- 
tional-level evaluation of the coverage of a recently 
introduced universal treatment programme for de- 
pression in primary care compared independently col- 
lected national-level cross-sectional survey data 
obtained before and after the programme was imple- 
mented. Respondents were asked if they had seen a 
doctor in connection with a depressive episode in the 
past 12 months. The study therefore estimated the 
increase in coverage that happened after the national 
programme was introduced, which may have been 
due to factors other than the programme itself. 

Estimation of the denominator: target population 

The methods used to measure the denominator, the 
total population in need of services, varied between 
studies. Three studies used nationally representative 
surveys to estimate the prevalence of the dis- 
order, ^^'^^'^^ two of which used regionally disaggre- 
gated prevalence figures to provide regional 
estimates of the target population. ^^'^^ Only one of 
these studies recognized that not all of those diag- 
nosed with a mental disorder are in need of, or 
seek, services. A national-level evaluation of the 
Improving Access to Psychological Therapies (lAPT) 
programme in England determined the size of their 
target population based not only on prevalence of the 
disorder, but on the other factors that determine ser- 
vice utilization.^'^ The authors' estimated: that of the 



six million people in England with common mental 
disorders (CMD) estimated from a national preva- 
lence survey, only half will seek treatment; that a 
further half will be diagnosed; and that, of these, 
70% will opt for psychological therapy. Therefore of 
the total population of six million people with CMD 
in England, only 15% (900 000 per year) are likely to 
actually use psychological therapy and represent the 
target population that the programme aims to reach 
in the first 5 years. Programme coverage was esti- 
mated as the proportion of this target to which the 
programme delivered services. 

An evaluation of a methadone treatment pro- 
gramme in China used a local register of opiate ad- 
dicts kept by the police department to estimate the 
size of the target population in need of treatment. 
Although no information was given as to the reliabil- 
ity and validity of these data which have the potential 
to be highly biased, nor as to its completeness in 
terms of the proportion of addicts registered with 
the police, comparing the number of addicts known 
to local authorities with those receiving treatment is 
at least a good indicator of the number of people with 
a known disorder that the programme could realistic- 
ally aim to reach. 

One study used an estimate of the percentage of the 
population who are expected to need services, based 
on prevalence rates taken from a review of the global 
literature^^ rather than from Italy where the pro- 
gramme was located. This study, and one other 
from Italy,^^ estimate total population coverage 
whereby the rate of service utilization taken from pa- 
tient case registers maintained by the programme is 
calculated per 100 000 of the total population, not the 
total population of people estimated to have a mental 
disorder. 

Methods to estimate equity of contact coverage 

Five of the seven studies disaggregated their contact 
coverage figures by the socio-demographic character- 
istics of service users such as gender, age and socio- 
demographic status, in order to estimate equity of 
contact coverage. ^'^'^^'^^"^^ However, only the evalu- 
ation of the national depression treatment programme 
in Chile compared the socio-demographic profile of 
service users with that of the total population of 
people with mental disorders in order to determine 
whether access to the programme was equitable. 

Methods to estimate effective coverage 

None of the studies reported effective coverage. The 
evaluation of I APT in England was the only study to 
report the impact of the programme on patient out- 
comes. The authors did not use this information to 
calculate effective coverage, but we were able to cal- 
culate this from data presented by the study. In 2012, 
lAPT treated two-thirds (9.68% out of 15%) of the 
900 000 cases of CMD that the programme aims to 
treat every year. Of these, 60% completed the course 
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Box 1 Directions for future research 

• Very few evaluations of mental health programmes include estimates of coverage, with no studies found 
in lower middle- and low-income countries. Current data are inadequate for planning. 

• The estimation of contact coverage requires a numerator of routinely collected service utilization data to 
be compared with a denominator of the number of people in the catchment area who require services. 

• Current gaps in national level prevalence data need to be filled by well-conducted population-based 
surveys in order to provide accurate estimates of the denominator. 

• New methods are required to more accurately estimate the proportion of people diagnosed with a mental 
disorder who are in need of services. This group should form the target population which the programme 
aims to cover, not all those diagnosed with a mental disorder. 

• Estimates of coverage should be broken down by factors such as gender and socioeconomic status to 
determine whether programme coverage is equitable. 

• The estimation of effectiveness coverage involves adjusting the contact coverage of a programme by the 
effectiveness of the programme on patient outcomes, to determine the proportion of those in need of 
services who benefit from them. 

• Evaluations of mental health programmes in all settings should routinely incorporate measures of contact 
and effectiveness coverage to improve existing services and to inform efforts to scale up services in 
settings where there are none. 



of treatment (were adherent), and 45% of these re- 
covered. This is compared with a target recovery of 
50% estimated from clinical trials of psychological 
therapies, thereby requiring a further downward ad- 
justment for the difference between efficacy demon- 
strated in clinical trials and the effectiveness of the 
treatment delivered in routine care.^^ This results in 
an effective coverage of 35%. 

Discussion 

Main findings 

This systematic review of coverage estimates reported 
by evaluations of mental health programmes has 
highlighted the paucity of such evaluations, with 
none at all in lower middle- and low-income coun- 
tries. As reported elsewhere, there are major gaps in 
global knowledge of the prevalence of mental dis- 
orders, resulting in inadequate information for 
policy and planning. Box 1 outlines implications 
for future research. 

This lack of evidence is partly due to the methodo- 
logical difficulties of estimating contact coverage. To 
estimate the denominator of the population in need of 
services in a catchment area, estimates of the preva- 
lence of the disorder in the catchment population are 
needed. Collecting this information is beyond the 
scope of individual programmes, so routine data 
need to be utilized. In high-income countries, preva- 
lence data are sometimes available through routine 
national surveys such as the Adult Psychiatric 
Morbidity Survey in England^^ and the national 
Survey of Mental Health and Wellbeing in 
Australia.^ ^ In settings where national- or regional- 
level survey data may not be available, reference can 
be made to the estimated prevalence of mental dis- 
orders in the scientific literature, for example in the 



World Mental Health surveys and the national 
prevalence rates for different disorders calculated as 
part of the Global Burden of Disease 2010 study.^^ 
However, these prevalence data are patchy, with 
more complete prevalence data for North America 
and Australasia, highly variable data from Europe, 
Latin America and Asia Pacific and poor levels of 
data in other regions of Africa and Asia.^^ In addition, 
the prevalence estimates that do exist are often for 
specific regions of a country, sub-groups of the popu- 
lation or specific disorders, and may not represent the 
population which a specific programme seeks to serve. 
If no such prevalence estimates for the population 
exist, then the number treated can be compared 
with the total population size to provide a ratio of 
the number treated per 100 000, which remains an 
informative metric for planning services, tracking 
changes in service utilization over time and compar- 
ing programme coverage with other non-mental 
health programmes, though these cannot provide an 
estimate of the proportion of those in need who are 
receiving services. 

Furthermore, the methods for measuring the de- 
nominator for coverage are based on the assumption 
that all those who are diagnosed with a mental dis- 
order are in need of treatment. This assumption 
clearly needs to be tested as severity of illness, suit- 
ability of treatment provided by the programme, other 
sources of support available and patient preferences 
will all affect the need for or uptake of services. 
Only one study included in this review addressed 
this issue. The UK lAPT study^^ determined the size 
of their target population based not only on the 
prevalence of the disorder, but on the other factors 
that determine service utilization. Whereas using 
such a 'rule of thirds' to estimate those that the pro- 
gramme should realistically intend to cover is crude, it 
does at least reflect the fact that a programme should 
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not attempt to cover all those with the target disorder 
in the population it serves, but rather the fraction 
comprising those who are willing to receive and 
would benefit from the treatment provided by the 
programme. 

Measurement of the number of people using the 
service (numerator) requires routine monitoring data 
to be collected by programmes. We anticipate that 
most programmes could routinely collect these in 
the form of numbers of patients referred and treated. 
To assess the equity of coverage, basic socio-demo- 
graphic information is also needed such as age, 
gender, area of residence and ethnic group, ideally 
broken down by diagnosis. Such information can be 
routinely collected by Health Management 
Information Systems (HMIS), but it needs to be com- 
pared with data from community-based prevalence 
surveys to determine whether programme coverage 
is equitable compared with the socio-demographic 
profile of the target population. In addition, popula- 
tion based surveys which include questions on treat- 
ment received can provide estimates of service 
utilisation, if the treatment is ascribed to particular 
programmes. 

The estimation of effective coverage requires out- 
come data for those treated, which are much more 
intensive to measure. It is notable that none of the 
included studies reported effective coverage, though 
we were able to calculate this for one of the studies 
based on the data provided by the lAPT programme. 
However, many programmes in high-resource settings 
do routinely collect patient outcome data as part of 
their clinical records, and similar systems can be im- 
plemented in low-resource settings with minimal in- 
vestment and staff training. Examples include routine 
clinical information systems being implemented in 
community mental health programmes in Nigeria by 
the NGO CBM, and by the NGO Health Net TPO in 
Burundi, Indonesia, Nepal, Sri Lanka and Sudan. 

To be of further use for service planning and devel- 
opment, evaluations of programme coverage should 
include an analysis of the determinants of coverage, 
so that factors which increase coverage can be built 
into the design of programmes. Only one of the evalu- 
ations included in this review^^ estimated differences 
in coverage between treatment clinics and explored 
the reasons for these differences. The study showed 
that structural factors such as the length of opening 
hours and incentives for compliant clients were posi- 
tively associated with increased coverage. Such infor- 
mation, especially if incorporated into pilot 
evaluations of new programmes, could be used for 
service development and improvement. 

There is a separate but related literature on the 
'treatment gap' which seeks to estimate the lack of 
coverage of any form of treatment for specified 
mental health problems. ^'^^ The denominator is esti- 
mated in the same way, that is by using nationally 
representative community surveys to estimate the 



total number of people with mental disorders in the 
population (as determined by a diagnosis of a mental 
disorder using a screening or diagnostic tool). 
Concerning the numerator, for treatment gap studies 
this is the number of cases in need who receive no 
care or treatment for their condition, which is typic- 
ally measured via cross-sectional community surveys 
that ask respondents to recall any treatment for a 
particular disorder for which they screen positive in 
the survey. Such survey data do not yield information 
about the coverage or effectiveness of specific inter- 
ventions or programmes, thus the treatment gap dif- 
fers from the direct measurement of coverage 
associated with a specific programme. In short, re- 
peated measurement of the treatment gap can 
inform policy and planning by revealing changes in 
the proportion of people receiving no care, but needs 
to be supplemented with information on the effective 
coverage of individual programmes so as to determine 
which types of programmes produce the best and 
most equitable patient outcomes for the largest num- 
bers of people. 

Strengths and limitations of the review 

There are a number of limitations of the review. First, 
as the term 'coverage' was so rarely used in descrip- 
tions of programme evaluations, we did not use spe- 
cific search terms for coverage but rather manually 
searched all programme evaluations to determine 
whether they included coverage. Due to the scope of 
this exercise (56 000 studies located in a search not 
restricted by language) we had to restrict the review 
to English language publications. Second, we did not 
list individual disorders by name. This may have re- 
sulted in some evaluations of treatment programmes 
which are not consistently reported as mental dis- 
orders being missed (such as substance misuse or 
eating disorders). To mitigate these limitations, we 
used a wide-ranging search strategy including elec- 
tronic database and internet searches and contacting 
key organizations and experts in the field to identify 
both published and grey literature. We used broad 
search terms to capture the wider literature on 
mental health programme evaluation, and then 
hand- searched these results to locate evaluations of 
coverage. We are confident that the studies included 
in this review represent the scope of the English lan- 
guage literature on this topic. 



Conclusion 

Evaluating the contact coverage and the effective 
coverage of mental health programmes is essential 
in order to track efforts to scale up effective and 
equitable services for people with mental disorders, 
and has great potential to guide work to improve 
the quality of existing services. Estimates of contact 
coverage could be incorporated into the routine 
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monitoring of mental health programmes simply and 
cheaply by analysing data routinely collected by pro- 
grammes and comparing these with population preva- 
lence estimates from national surveys. Better 
estimates of programme contact and effective cover- 
age, the equity of coverage and the factors that affect 
coverage are needed to ensure that there is optimal 
investment of scarce resources into existing mental 
health programmes, and that new mental health ser- 
vices implemented in low-resource settings are de- 
signed to optimise the effective and equitable 
programme coverage of those in greatest need. 
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KEY MESSAGES 

• Evaluating both contact coverage and effective coverage of mental health programmes is essential in 
order to track efforts to scale up services for people with mental disorders. 

• We did a systematic review of mental health programme evaluations and found only seven evalu- 
ations of coverage. 

• These studies used routine clinical information or national community surveys to estimate service 
utilization (the numerator of coverage); and national surveys, case registers or estimates from other 
literature to estimate the population in need of services (the denominator of coverage). 

• Current evidence is inadequate for the needs of policy makers and planners, and better measures of 
programme contact and effective coverage need to be incorporated into routine evaluations of mental 
health programmes. 
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